Industrial robot system having sensor assembly

ABSTRACT

An industrial robot system includes an end effector connectable to a robot arm, a drive assembly, and a controller. The end effector includes a distal housing, a spindle assembly rotatable about a rotational axis, a drill bit rotatable about the rotational axis, and a sensor assembly. The sensor assembly includes a first light source, a second light source, and a photosensitive array. The first light source produces a first fan of light which is projected as a first line of light on the object surface. The second light source produces a second fan of light, which is projected as a second line of light on the object surface. The photosensitive array detects a first reflection line corresponding to the first line of light and a second reflection line corresponding to the second line of light.

BACKGROUND

Industrial robots are used to position and support tools for operationon large work pieces. Conventional robots have limitations on theaccuracy to position and hold a tool with respect to the work piece.

SUMMARY

An example of an industrial robot system having an increased accuracy inpositioning a tool with respect to a work piece includes an end effectorconnectable to a robot arm, a drive assembly operatively connected withthe robot arm, and a controller in electrical communication with thedrive assembly. The end effector includes a distal housing, a spindleassembly connected with the distal housing and rotatable about arotational axis, a drill bit connected with the spindle assembly androtatable about the rotational axis, and a sensor assembly mounted tothe distal housing offset from the rotational axis. The drive assembly,which is operatively connected with the robot arm, moves the robot armto locate the drill bit in the desirable location. The controller is inelectrical communication with the sensor assembly and the drive assemblyand is configured to transmit signals to the drive assembly to move therobot arm to adjust the rotational axis of the drill bit with respect toan object surface of the work piece.

The sensor assembly mounted to the distal housing of the end effectorincludes a first light source offset from the rotational axis, a secondlight source offset from the rotational axis and the first light source,and a photosensitive array offset from the rotational axis, the firstlight source, and the second light source. The first light source isconfigured to produce a first fan of light which is projected as a firstline of light on the object surface. The second light source isconfigured to produce a second fan of light, which is projected as asecond line of light on the object surface. The second fan of lightresides in a second plane, which is offset at an angle transverse to afirst plane in which the first fan of light resides. The photosensitivearray is positioned with respect to the first light source and thesecond light source to detect a first reflection line corresponding tothe first line of light and a second reflection line corresponding tothe second line of light.

A method for controlling an industrial robot includes moving a point ona robot arm adjacent to a nominal location within a robot frame. Themethod further includes determining x, y and z-coordinates, and R_(x)and R_(y) of the point on the robot arm with respect to an x-y planedefined by an object surface and a z-axis normal to the x-y plane. Therobot arm includes a drill bit rotatable about a rotational axis. R_(x)is an angle between the rotational axis and the z-axis in anx-direction. R_(y) is an angle between the rotational axis and thez-axis in a y-direction.

Determining the x, y and z-coordinates and the R_(x) and R_(y) of thepoint on the robot arm further includes projecting a first line of lightfrom a first light source onto the object surface, projecting a secondline of light from a second light source, which is offset from the firstlight source, onto the object surface, detecting a reflection of thefirst line of light on a photosensitive array to generate a firstreflection line, detecting a reflection of the second line of light onthe photosensitive array to generate a second reflection line, andcalculating the x, y and z-coordinates, and R_(x) and R_(y) of the pointon the robot arm. Both the first line of light and the second line oflight are projected from each respective light source onto the objectsurface without moving the point on the robot arm from adjacent thenominal location. The second line of light intersects the first line oflight at an intersection point on the object plane where the rotationalaxis intersects the object plane. The photosensitive array is offsetfrom the first light source and the second light source. Calculating thex, y and z-coordinates and R_(x) and R_(y) of the point on the robot armis based on the intersection point and an angle between the firstreflection line and the second reflection line.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a longitudinal schematic partial cross-sectional view of anend effector connected to a robot arm of an industrial robot.

FIG. 2 is a view taken generally along line 2-2 through the end effectordepicted in FIG. 1.

FIG. 3 is a schematic depiction of components of a sensor assembly ofthe end effector depicted in FIG. 1.

FIG. 4 is a schematic depiction of an image captured by a photosensitivearray of the end effector depicted in FIG. 1.

FIG. 5 is another schematic depiction of an image captured by thephotosensitive array of the end effector depicted in FIG. 1.

FIG. 6 is another schematic depiction of an image captured by thephotosensitive array of the end effector depicted in FIG. 1

DETAILED DESCRIPTION

FIG. 1 depicts an industrial robot system including an end effector 10connectable to a robot arm 12 (schematically depicted in FIG. 1) of anindustrial robot such as one manufactured by KUKA Roboter GmbH. Therobot system further includes a drive assembly 14 and a controller 16,each of which are only schematically depicted in FIG. 1. The controller16 is in electrical communication with the drive assembly 14. The driveassembly 14, which can be similar to drive assemblies that are known inthe art, is operatively connected with the robot arm 12 in aconventional manner for moving the robot arm 12 throughout the robotframe. The controller 16 transmits signals to the drive assembly 14 tomove the robot arm 12 to adjust and to move the robot arm within therobot frame.

The end effector 10 includes a changer tool side 18 that couples the endeffector 10 to a complementary master side of a tool changer 20 fixed toan end of the robot arm 12. The changer tool side 18 of the end effector10 can provide electric signals from the controller 16 to the endeffector 10 as well as air pressure and coolant via connections andlines that are not shown. The end effector 10 further includes a distalhousing 22. The distal housing 22 is hollow and can be generallycone-shaped and may also be referred to as a nose or cone of the endeffector. The distal housing 22 includes a distal opening 24. The distalhousing 22 further includes an internal surface 26 and an externalsurface 28.

The end effector 10 further includes a spindle assembly 32 connectedwith the distal housing 22 and rotatable about a rotational axis 34. Thedistal housing 22 is centered with respect to the rotational axis 34 andthe distal opening 24 is concentric with the rotational axis 34. Thespindle assembly 32 is driven by a motor or other drive assembly, whichcan be conventional and is, therefore, not shown. The spindle assembly32 carries an automatic tool clamp 36. Many of the components of the endeffector are not shown, and these components can be conventionalcomponents and/or components further described in U.S. 2012/0020756 A1,which is incorporated by reference herein.

The end effector 10 further includes a drill bit 38 connected with thespindle assembly 32 and rotatable about the rotational axis 34. Asillustrated, the tool clamp 36 couples the drill bit 38 to the spindleassembly 32. The spindle assembly 32 and the drill bit 38 alsotranslates along the rotational axis 34, which allows the drill bit 38to extend through the distal opening 24 in the distal housing 22 whenthe drill bit 38 is to operate on a work piece.

With reference to FIGS. 1 and 2, the end effector 10 further includes asensor assembly 50 mounted to the distal housing 22 offset from therotational axis 34. As more clearly visible in FIG. 2, the sensorassembly 50 generally includes a first light source 52, a second lightsource 54, and a camera 56. As more clearly visible in FIG. 1, thecamera 56 includes a lens 58 and a photosensitive array 60. FIG. 1schematically depicts the camera 56 and the second light source 54attached to the internal surface 26 of the distal housing 22.

With reference back to FIG. 2, the sensor assembly 50 further includes aframe 70 having a curved base wall 72 that is complementary in shape andconfiguration to the internal surface 26 of the distal housing 22. Theframe 70 connects to the distal housing 22 and contacts the innersurface 26 of the distal housing. The frame 70 includes a first lightsource receptacle 74, a second light source receptacle 76, and a camerareceptacle 78. The first light source receptacle 74 receives the firstlight source 52, which fixes the location of the first light source 52with respect to the rotational axis 34 and the distal opening 24.Similarly, the second light source receptacle 76 receives second lightsource 54, which fixes the location of a second light source 54 withrespect to the rotational axis 34 and the distal opening 24. Also, thecamera receptacle 78 receives the camera 56, which fixes the location ofthe photosensitive array 60 (FIG. 1) with respect to the rotational axis34 and a distal opening 24.

With reference to FIG. 3, the first light source 52 is configured toproduce a first fan of light 82, which is projected as a first line oflight 84 on an object surface 86 of a work piece. The work piece is themanufactured component that is being operated on by the industrialrobot. The second light source 54 is configured to produce a second fanof light 88, which is projected as a second line of light 92 on theobject surface 86. The first fan of light 82 resides in a first plane,and the second fan of light 88 resides in a second plane that is offsetat an angle transverse to the first plane. As illustrated in FIG. 3, thefirst plane is perpendicular to the second plane such that the firstlight of line 84 is perpendicular to the second line of light 92. Thefirst line of light 84 need not be perpendicular to the second line oflight 92. Each of the first light source 52 and the second light source54 is a structured light source capable of producing the aforementionedfans of light. Lasers and/or linear LEDs are preferred. Other lightsources that can produce the desired lines on the object surface canalso be used.

In the illustrated embodiment, the first light source 52 is positionedwith respect to the distal opening 24 (FIG. 1) such that the first fanof light 82 is projected through the distal opening 24. Similarly, thesecond light source 54 is positioned with respect to the distal opening24 such that the second fan of light 88 is projected through the distalopening. Also, the photosensitive array 60 is positioned with respect tothe distal opening 24 such that light reflected from the first line oflight 84 and the second line of light 92 passes through the distalopening 24 en route to the photosensitive array 60.

As depicted in FIGS. 1 and 2, the sensor assembly 50 is located withinthe hollow distal housing 22. If desired, the sensor assembly 50 orcomponents thereof could be located external of the distal housing 22,for example, by mounting to the external surface 28 of the distalhousing. Positioning of the sensor assembly 50 in the end effector 10and/or adjacent the rotational axis 34 and the distal opening 24obviates the need for the robot system to move a camera to a nominallocation to obtain an image of the work surface and then move a drillbit back to the nominal location to perform the working operation. Thesensor assembly 50, which obtains images of the object surface 86 (FIG.3,) is near enough the rotational axis that the number of movements ofthe robot arm 12 is reduced.

With reference back to FIG. 3, the first light source 52 is positionedwith respect to the rotational axis 34 and the second light source 54 ispositioned with respect to the rotational axis 34 and the first lightsource 52 such that the first fan of light 82 intersects the second fanof light 88 along the rotational axis 34. As mentioned above, the firstlight source 52 is positioned with respect to the second light source 54such that the first fan of light 82 is perpendicular to the second fanof light 88; however, such a configuration is not necessary. The firstlight source 52 is offset from a center point 94 of the photosensitivearray 60 at angle A. The second light source 54 is offset from thecenter point 94 of the photosensitive array 60 at angle B. In theillustrated embodiment, angle A is equal in magnitude but opposite indirection from angle B.

The controller 16 is configured to receive a signal from the sensorassembly 50. The controller 16 measures normalcy of the rotational axis34 with respect to the object surface 86. The controller 16 operates thedrive assembly 14 to adjust the robot arm 12 based on the measurednormalcy of the rotational axis 34 with respect to the object surface.The controller determines R_(x) and R_(y) of the rotational axis 34 withrespect to a z-axis 102, which is normal to an x-y plane 104 on theobject surface 86 where the x-y plane 104 is normal to the z-axis. R_(x)is an angle between the rotational axis 34 and the z-axis 102 in anx-direction. R_(y) is an angle between the rotational axis 34 and thez-axis 102 in a y-direction.

With reference to FIG. 4, the controller determines R_(x) and R_(y)based on a first reflection line 112 detected by the photosensitivearray 60 and a second reflection line 114 detected by the photosensitivearray. FIG. 4 depicts an image 116 captured by the photosensitive array60. The image 116 can be displayed on a conventional display (not shown)that is in communication with the controller 16. The photosensitivearray 60 can be a conventional CCD sensor having a photosensitive arraymatrix that can resolve the received image into a plurality of pixels toallow for calculations in an x, y and z-coordinate system. In theillustrated embodiment, the first light source 52 is configured withrespect to the second light source 54 and the rotational axis 34 suchthat the first line of light 84 and the second line of light 92 aredisposed perpendicular to one another. Due to this configuration, if therotational axis 34 were normal to the object surface 86 (such that therotational axis 34 is coincident with the z-axis 102 in FIG. 3) then thefirst reflection line 112 would be perpendicular to the secondreflection line 114. The angular offset that the first reflection line112 is off from a horizontal line 118 through an intersection point 122for the first reflection line 112 and the second reflection line 114correlates to R. The angular offset that the second reflection line 114is offset from a vertical line 124 that also intersects the intersectionpoint 122 correlates to R_(y). These angles can be fed to the controller16 which can in turn operate the drive assembly 14 to maneuver the robotarm 12 to correct for the offset of normalcy of the rotational axis 34with respect to the x-y plane 104 on the object surface 86.

The controller 16 also determines x, y and z-coordinates for a point 130on the rotational axis 34 with respect to the x-y plane and the z-axis.The point 130 can be a distal end of the drill bit 38 when the drill bitis in the retracted position, which is shown in FIG. 1. The center point94 on the photosensitive array 60 is offset from the point 130 on therotational axis 34 a predetermined distance d_(z) measured parallel tothe rotational axis 34. The center point 94 is also offset from thepoint 130 on the rotational axis 34 a predetermined distance d_(x)measured in the x-direction, and is offset from the point 130 apredetermined distance d_(y) measured in the y-direction. Also, theorientation of the photosensitive array 60 with respect to the first fanof light 82 and the second fan of light 88 is known. These knownrelationships allow for the location of the point 130 on the robot arm12 with respect to the x-y plane and the z-axis based on knowntrigonometric equations and the type (size) of CCD array used for thephotosensitive array 60.

In operation, the point 130 on the robot arm 12 is moved adjacent to anominal location within the robot frame. Movement of the robot arm 12toward the nominal location can be performed by the controller 16providing signals to the drive assembly 14 to drive the robot arm to aknown location, which is the nominal location. With reference to FIG. 3,the nominal location can be where the point 130 on the robot arm 12 isaligned with a point on the object surface 86 but offset in az-direction a minimal distance from the x-y plane 104. With reference toFIG. 5, the x and y coordinates of the nominal location could correspondto what is thought to be a center (hereinafter “nominal center”) 150 ofa circular hole 152 formed in the object surface. FIG. 5 depicts animage 154 captured by the photosensitive array 60 when the rotationalaxis 34 is normal to the x-y plane 104 on the object surface 86. FIG. 5depicts the circular hole 152 with dashed lines. The circular hole 152would not be visible in the captured image 154, but is shown for thesake of clarity. With the point 130 (FIG. 1) on the robot arm 12 movedto the nominal location, and the normalcy of the rotational axis 34being within a predetermined tolerance (so that the rotational axis 34can be assumed to be normal to the x-y plane 104), the x, y andz-coordinates for the point 130 on the robot 12 can be determined due tothe known nominal location. The normalcy of the rotational axis 34 withrespect to the x-y plane 104 can be determined as described above.

With the point 130 on the robot arm 12 positioned at the nominal center150, a first line of light similar to the first line of light 84 shownin FIG. 3 is projected onto the object surface 86. A first reflectionline 156 is detected by the photosensitive array 60, which correspondsto a first line of light. With the point 130 on the robot arm 12positioned at the nominal center 150, a second line of light similar tothe second line of light 92 shown in FIG. 3 is projected onto the objectsurface 86. A second reflection line 158 is detected by thephotosensitive array 60, which corresponds to a second line of light.Since the rotational axis 34 (FIG. 1) is normal to the x-y plane 104(FIG. 3), the first reflection line 156 is perpendicular to the secondreflection line 158 and the first reflection line 156 would intersectthe second reflection line 158 if the hole was not in the work surface86 (FIG. 3).

Since there is the hole (denoted by the circle 152 in FIG. 5) in thework surface 86, there will be a break in the first reflection line 156and the second reflection line 158. Accordingly, four points, e.g.,break points 156 a, 156 b, 158 a and 158 b, for the hole 152 are known.Since the hole 152 is circular, curve fitting and/or least squareanalysis can be used to determine the periphery of the hole. With theperiphery of the circular hole 152 known, the true center 162 of thecircular hole 152 can be determined and the offset between the nominalcenter 150, i.e. what was thought to be the center of the hole 152, andthe true center 162 of the hole 152 can be determined based on thenumber of pixels between nominal center 150 and the true center 162 inthe x and y directions.

If the rotational axis 34 aligns with the true center 162 of the hole152 and is normal to the x-y plane 104, the controller 16 can deliver asignal to the spindle assembly 32 (FIG. 1) to rotate and translate thedrill bit 38 (FIG. 1) to drill into the hole with the drill bit. If therotational axis 34 is not aligned with the true center 162 of the hole152 or is not normal to the x-y plane 104, then the controller 16 sendsa signal to the drive assembly 14 to move the robot arm 12 and thenrepeat the detecting of the center of the hole and checking to seewhether the rotational axis aligns with the true center and is normal tothe x-y plane until each of these conditions are satisfied. As such, therobot arm 12 can bring the drill bit 38 into proper location withrespect to a pre-drilled hole and drill out the hole normal to theobject surface of the work piece. The first line of light and the secondline of light, which are similar to the first line of light 84 and thesecond line of light 92 respectively in FIG. 3, can be projected ontothe object surface 86 (FIG. 3) without moving the point 130 on the robotarm 12 from adjacent the nominal location, i.e., the nominal center 150.The increases the speed and accuracy of the system as compared to knownrobot systems.

The robot arm 12 and the drill bit 38 can also be used to drill acounterbore into the work surface 86. With reference to FIG. 5, when therotational axis 34 aligns with the true center 162 of the hole 152 andis normal to the x-y plane 104, the controller 16 can deliver a signalto the spindle assembly 32 (FIG. 1) to rotate and translate the drillbit 38 (FIG. 1) to drill a counterbore into the hole with the drill bit.FIG. 6 depicts an image 178 captured by the photosensitive array 60 of acounterbore formed in the object surface 86. The counterbore has anouter diameter (OD) 182 and an inner diameter (ID) 184. FIG. 6 depictsthe image 178 captured when the rotational axis 34 is normal to the x-yplane 104 (FIG. 3) on the object surface 86 (FIG. 3). FIG. 6 depicts theOD 182 and the ID 184 with dashed lines. The OD 182 and the ID 184 wouldnot be visible in the captured image 178, but is shown for the sake ofclarity. With the point 130 on the robot arm 12 positioned at a center186 of the OD 182 and the ID 184 and normal to the x-y plane 104, afirst line of light similar to the first line of light 84 shown in FIG.3 is projected onto the object surface 86. A first reflection line 192is detected by the photosensitive array 60, which corresponds to a firstline of light. With the point 130 on the robot arm 12 positioned at thecenter 186 of the OD 182 and the ID 184 and normal to the x-y plane 104,a second line of light similar to the second line of light 92 shown inFIG. 3 is projected onto the object surface 86. A second reflection line194 is detected by the photosensitive array 60, which corresponds to asecond line of light.

The first reflection line 192 changes direction at the OD 182 at points192 a and 192 b. This is due to the change in the z-direction betweenthe OD 182 and the ID 184 of the counterbore. The first reflection line192 breaks at points 192 c and 192 d on the ID 184. Similarly, thesecond reflection line 194 changes direction at the OD 182 at points 194a and 194 b. . The second reflection line 194 breaks at points 194 c and194 d on the ID 184. Similar curve fitting and least squares analysiscan be performed to determine the respective ID 184 and OD 182 based onthese lines of reflection. The diameter of the OD 182 and the ID 184 canbe measured based on the curve fitting and least squares analysis. Thedepth of the counterbore in the z-direction can be determined based onthe change in direction of the first line of reflection 192 at the OD182 and based on the change in direction of the second line ofreflection 194 at the OD.

The nominal location can also provide a datum point for the robot. Insuch an instance, the nominal location is typically one of a pluralityof nominal locations each corresponding to a respective hole or fastenerin the object surface 86. A center of the first hole or fastener at afirst nominal location is detected in a manner similar to the detectionof the true center 162 of the hole 152 described above with reference toFIG. 5. If the rotational axis 34 is normal to the x-y plane 104 (FIG.3), then the controller 16 calculates and records the x, y andz-coordinates for the true center 162 with respect to the nominal center150. If the rotational axis 34 is not normal to the x-y plane 104 (FIG.3), then the robot arm 12 is moved and the detecting of the center ofthe holes and checking to see if the rotational axis is normal to thex-y plane is repeated. Once the true center 162 is detected, the x, yand z-coordinates are recorded and the point 130 on the robot arm 12moves to the next nominal location and repeats these steps. Once therobot system has repeated the steps at at least three different nominallocations, the entire robot frame can then be shifted based on theoffset of the nominal locations from the true locations. With the robotframe shifted in the Cartesian coordinates, the robot arm could then bemoved to other desired locations and other holes and other holes can bedrilled into the work surface.

It will be appreciated that various of the above-disclosed and otherfeatures and functions, or alternatives or varieties thereof, may bedesirably combined into many other different systems or applications.Also that various presently unforeseen or unanticipated alternatives,modifications, variations or improvements therein may be subsequentlymade by those skilled in the art which are also intended to beencompassed by the following claims.

The invention claimed is:
 1. A method for controlling an industrialrobot comprising: moving a point on a robot arm adjacent to a nominallocation within a robot frame, wherein the robot arm includes a drillbit rotatable about a rotational axis; and determining x, y andz-coordinates, and R_(x) and R_(y) of the point on the robot arm withrespect to an x-y plane defined by an object surface, wherein a z-axisis normal to the x-y plane, wherein R_(x) is a first angle between therotational axis and the z-axis in an x-direction and R_(y) is a secondangle between the rotational axis and the z-axis in a y-direction,wherein determining the x, y and z-coordinates, and R_(x) and R_(y) ofthe point on the robot arm further includes projecting a first line oflight from a first light source onto the object surface without movingthe point on the robot arm from adjacent the nominal location;projecting a second line of light from a second light source, which isoffset from the first light source, onto the object surface withoutmoving the point on the robot arm from adjacent the nominal location,wherein the second line of light intersects the first line of light atan intersection point on the object plane where the rotational axisintersects the object plane; detecting a reflection of the first line oflight on a photosensitive array offset from the first light source andthe second light source to generate a first reflection line; detecting areflection of the second line of light on a photosensitive array offsetfrom the first light source and the second light source to generate asecond reflection line; and calculating the x, y and z-coordinates, andR_(x) and R_(y) of the point on the robot arm based on the intersectionpoint and an angle between the first reflection line and the secondreflection line.
 2. The method of claim 1, wherein the nominal locationcorresponds to a hole in the object surface, the method furthercomprising: (a) detecting a center of the hole based on a change indirection or break in the first reflection line and the secondreflection line; (b) if the rotational axis aligns with the center ofthe hole and is normal to the x-y plane, drilling into the hole with thedrill bit, (c) if the rotational axis is not aligned with the center ofthe hole or is not normal to the x-y plane, then moving the robot armand repeating steps (a) and (b).
 3. The method of claim 2, whereindetecting the center of the hole further includes performing a curvefitting or least squares analysis based on break points in the firstreflection line and the second reflection line.
 4. The method of claim1, wherein the nominal location is one of a plurality of nominallocations each corresponding to a respective hole or a fastener in theobject surface, the method further comprising: (a) detecting a center ofa first hole or fastener at a first nominal location of the plurality ofnominal locations based on a change in direction or break in the firstreflection and the second reflection line; (b) if the rotational axis isnormal to the x-y plane, calculating and recording the x, y andz-coordinates for the center, (c) if the rotational axis is not normalto the x-y plane, then moving the robot arm and repeating steps (a) and(b).